verification and validation
Fighting AI with AI: Leveraging Foundation Models for Assuring AI-Enabled Safety-Critical Systems
Mavridou, Anastasia, Gopinath, Divya, Păsăreanu, Corina S.
The integration of AI components, particularly Deep Neural Networks (DNNs), into safety-critical systems such as aerospace and autonomous vehicles presents fundamental challenges for assurance. The opacity of AI systems, combined with the semantic gap between high-level requirements and low-level network representations, creates barriers to traditional verification approaches. These AI-specific challenges are amplified by longstanding issues in Requirements Engineering, including ambiguity in natural language specifications and scalability bottlenecks in formalization. We propose an approach that leverages AI itself to address these challenges through two complementary components. REACT (Requirements Engineering with AI for Consistency and Testing) employs Large Language Models (LLMs) to bridge the gap between informal natural language requirements and formal specifications, enabling early verification and validation. SemaLens (Semantic Analysis of Visual Perception using large Multi-modal models) utilizes Vision Language Models (VLMs) to reason about, test, and monitor DNN-based perception systems using human-understandable concepts. Together, these components provide a comprehensive pipeline from informal requirements to validated implementations.
Grand Challenges in the Verification of Autonomous Systems
Leahy, Kevin, Asgari, Hamid, Dennis, Louise A., Feather, Martin S., Fisher, Michael, Ibanez-Guzman, Javier, Logan, Brian, Olszewska, Joanna I., Redfield, Signe
Autonomous systems use independent decision-making with only limited human intervention to accomplish goals in complex and unpredictable environments. As the autonomy technologies that underpin them continue to advance, these systems will find their way into an increasing number of applications in an ever wider range of settings. If we are to deploy them to perform safety-critical or mission-critical roles, it is imperative that we have justified confidence in their safe and correct operation. Verification is the process by which such confidence is established. However, autonomous systems pose challenges to existing verification practices. This paper highlights viewpoints of the Roadmap Working Group of the IEEE Robotics and Automation Society Technical Committee for Verification of Autonomous Systems, identifying these grand challenges, and providing a vision for future research efforts that will be needed to address them.
A Joint Approach Towards Data-Driven Virtual Testing for Automated Driving: The AVEAS Project
Eisemann, Leon, Fehling-Kaschek, Mirjam, Forkert, Silke, Forster, Andreas, Gommel, Henrik, Guenther, Susanne, Hammer, Stephan, Hermann, David, Klemp, Marvin, Lickert, Benjamin, Luettner, Florian, Moss, Robin, Neis, Nicole, Pohle, Maria, Schreiber, Dominik, Sowa, Cathrina, Stadler, Daniel, Stompe, Janina, Strobelt, Michael, Unger, David, Ziehn, Jens
With growing complexity and responsibility of automated driving functions in road traffic and growing scope of their operational design domains, there is increasing demand for covering significant parts of development, validation, and verification via virtual environments and simulation models. If, however, simulations are meant not only to augment real-world experiments, but to replace them, quantitative approaches are required that measure to what degree and under which preconditions simulation models adequately represent reality, and thus allow their usage for virtual testing of driving functions. Especially in research and development areas related to the safety impacts of the "open world", there is a significant shortage of real-world data to parametrize and/or validate simulations - especially with respect to the behavior of human traffic participants, whom automated vehicles will meet in mixed traffic. This paper presents the intermediate results of the German AVEAS research project (www.aveas.org) which aims at developing methods and metrics for the harmonized, systematic, and scalable acquisition of real-world data for virtual verification and validation of advanced driver assistance systems and automated driving, and establishing an online database following the FAIR principles.
Towards Mechatronics Approach of System Design, Verification and Validation for Autonomous Vehicles
Samak, Chinmay Vilas, Samak, Tanmay Vilas, Krovi, Venkat
Modern-day autonomous vehicles are increasingly becoming complex multidisciplinary systems composed of mechanical, electrical, electronic, computing and information sub-systems. Furthermore, the individual constituent technologies employed for developing autonomous vehicles have started maturing up to a point, where it seems beneficial to start looking at the synergistic integration of these components into sub-systems, systems, and potentially, system-of-systems. Hence, this work applies the principles of mechatronics approach of system design, verification and validation for the development of autonomous vehicles. Particularly, we discuss leveraging multidisciplinary co-design practices along with virtual, hybrid and physical prototyping and testing within a concurrent engineering framework to develop and validate a scaled autonomous vehicle using the AutoDRIVE ecosystem. We also describe a case-study of autonomous parking application using a modular probabilistic framework to illustrate the benefits of the proposed approach.
Automated driving system assessment - Siemens partners with IVEX
It will take some time before we can carelessly read the newspaper in the back seat of our self-driving car. Nevertheless, the automotive industry is working hard to push the limits in the development of vehicles with higher levels of autonomy. One of the major challenges the industry is facing is how to test an automated driving system. They also need to validate that autonomous vehicles are safe enough to be released on the public road. The verification and validation (V&V) process of automated driving systems is a challenging task, requiring a complex setup of tests.
Peugeot, Altran team for driverless car testing
Altran's parent, Capgemini, is combining the engineering skills of Altran with its data infrastructure and has launched a service for end to end support for validation and verification of driverless car systems. Altran also includes Cambridge Consultants in the UK which has been developing AI and sensor technologies for autonomous systems. The validation technology is being used by the maker of Citroen, Peugeot and Opel-Vauxhall cars to manage thousands of petabytes of data from testing the next generation of driverless cars. "We wanted to work with Capgemini and Altran because of their strong skills in data oriented and cloud- based projects. Participating in a European innovation project for the automotive industry in the field of the connected and autonomous car is very challenging. This collaboration enables us to complete our data collection and processing on schedule, and helps us to deploy innovative solutions for data analysis methods on a hybrid-cloud based solution," said Jean-Louis Sauvaget, Research & Development Division, Expert car data acquisition and post processing for customer usage in Groupe PSA.
Deep Learning & Software Engineering: State of Research and Future Directions
Devanbu, Prem, Dwyer, Matthew, Elbaum, Sebastian, Lowry, Michael, Moran, Kevin, Poshyvanyk, Denys, Ray, Baishakhi, Singh, Rishabh, Zhang, Xiangyu
The advent of deep learning (DL) has fundamentally changed the landscape of modern software. Generally, a DL system is comprised of several interconnected computational units that form "layers" which perform mathematical transformations, according to sets of learnable parameters, on data passing through them. These architectures can be "trained" for specific tasks by updating the parameters according to a model's performance on a labeled set of training data. DL represents a fundamental shift in the manner by which machines learn patterns from data by automatically extracting salient features for a given computational task, as opposed to relying upon human intuition. These DL systems can be viewed as an inflection point for software development, as they enable new capabilities that cannot be realized cost-effectively through "traditional" software wherein the behavior of a program must be specified analytically.
Great Powers Must Talk to Each Other About AI
Imagine an underwater drone armed with nuclear warheads and capable of operating autonomously. Now imagine that drone has lost its way and wandered into another state's territorial waters. Russia aims to field just such a drone by 2027, CNBC reported last year, citing those familiar with a U.S. intelligence assessment. Known as Poseidon, the drone will be nuclear-armed and nuclear-powered. While the dynamics of artificial intelligence and machine learning, or ML, research remain open and often collaborative, the military potential of AI has intensified competition among great powers.